File retrieval during a legacy storage system to dispersed storage network migration

ABSTRACT

A method begins by a processing module receiving a retrieval request for a file and determining whether the file is being migrated from a legacy storage system to a dispersed error coding storage system. The method continues with the processing module determining a retrieval option for the file when the file is being migrated from the legacy storage system to the dispersed error coding storage system. The method continues with the processing module retrieving the file, based on the retrieval option, in at least one of a legacy format from the legacy storage system and a plurality of sets of encoded data slices from the dispersed error coding storage system.

CROSS REFERENCE TO RELATED PATENTS

This patent application is claiming priority under 35 USC §119 to aprovisionally filed patent application entitled “MEMORY UTILIZATION IN ADISPERSED STORAGE NETWORK,” having a provisional filing date of Nov. 1,2010, and a provisional Ser. No. 61/408,971, pending, which is herebyincorporated herein by reference in its entirety and made part of thepresent U.S. Utility patent application for all purposes.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

NOT APPLICABLE

INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A COMPACT DISC NOTAPPLICABLE BACKGROUND OF THE INVENTION

1. Technical Field of the Invention

This invention relates generally to computing systems and moreparticularly to data storage solutions within such computing systems.

2. Description of Related Art

Computers are known to communicate, process, and store data. Suchcomputers range from wireless smart phones to data centers that supportmillions of web searches, stock trades, or on-line purchases every day.In general, a computing system generates data and/or manipulates datafrom one form into another. For instance, an image sensor of thecomputing system generates raw picture data and, using an imagecompression program (e.g., JPEG, MPEG, etc.), the computing systemmanipulates the raw picture data into a standardized compressed image.

With continued advances in processing speed and communication speed,computers are capable of processing real time multimedia data forapplications ranging from simple voice communications to streaming highdefinition video. As such, general-purpose information appliances arereplacing purpose-built communications devices (e.g., a telephone). Forexample, smart phones can support telephony communications but they arealso capable of text messaging and accessing the internet to performfunctions including email, web browsing, remote applications access, andmedia communications (e.g., telephony voice, image transfer, musicfiles, video files, real time video streaming, etc.).

Each type of computer is constructed and operates in accordance with oneor more communication, processing, and storage standards. As a result ofstandardization and with advances in technology, more and moreinformation content is being converted into digital formats. Forexample, more digital cameras are now being sold than film cameras, thusproducing more digital pictures. As another example, web-basedprogramming is becoming an alternative to over the air televisionbroadcasts and/or cable broadcasts. As further examples, papers, books,video entertainment, home video, etc. are now being stored digitally,which increases the demand on the storage function of computers.

A typical computer storage system includes one or more memory devicesaligned with the needs of the various operational aspects of thecomputer's processing and communication functions. Generally, theimmediacy of access dictates what type of memory device is used. Forexample, random access memory (RAM) memory can be accessed in any randomorder with a constant response time, thus it is typically used for cachememory and main memory. By contrast, memory device technologies thatrequire physical movement such as magnetic disks, tapes, and opticaldiscs, have a variable response time as the physical movement can takelonger than the data transfer, thus they are typically used forsecondary memory (e.g., hard drive, backup memory, etc.).

A computer's storage system will be compliant with one or more computerstorage standards that include, but are not limited to, network filesystem (NFS), flash file system (FFS), disk file system (DFS), smallcomputer system interface (SCSI), internet small computer systeminterface (iSCSI), file transfer protocol (FTP), and web-baseddistributed authoring and versioning (WebDAV). These standards specifythe data storage format (e.g., files, data objects, data blocks,directories, etc.) and interfacing between the computer's processingfunction and its storage system, which is a primary function of thecomputer's memory controller.

Despite the standardization of the computer and its storage system,memory devices fail; especially commercial grade memory devices thatutilize technologies incorporating physical movement (e.g., a discdrive). For example, it is fairly common for a disc drive to routinelysuffer from bit level corruption and to completely fail after threeyears of use. One solution is to a higher-grade disc drive, which addssignificant cost to a computer.

Another solution is to utilize multiple levels of redundant disc drivesto replicate the data into two or more copies. One such redundant driveapproach is called redundant array of independent discs (RAID). In aRAID device, a RAID controller adds parity data to the original databefore storing it across the array. The parity data is calculated fromthe original data such that the failure of a disc will not result in theloss of the original data. For example, RAID 5 uses three discs toprotect data from the failure of a single disc. The parity data, andassociated redundancy overhead data, reduces the storage capacity ofthree independent discs by one third (e.g., n−1=capacity). RAID 6 canrecover from a loss of two discs and requires a minimum of four discswith a storage capacity of n−2.

While RAID addresses the memory device failure issue, it is not withoutits own failures issues that affect its effectiveness, efficiency andsecurity. For instance, as more discs are added to the array, theprobability of a disc failure increases, which increases the demand formaintenance. For example, when a disc fails, it needs to be manuallyreplaced before another disc fails and the data stored in the RAIDdevice is lost. To reduce the risk of data loss, data on a RAID deviceis typically copied on to one or more other RAID devices. While thisaddresses the loss of data issue, it raises a security issue sincemultiple copies of data are available, which increases the chances ofunauthorized access. Further, as the amount of data being stored grows,the overhead of RAID devices becomes a non-trivial efficiency issue.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

FIG. 1 is a schematic block diagram of an embodiment of a computingsystem in accordance with the invention;

FIG. 2 is a schematic block diagram of an embodiment of a computing corein accordance with the invention;

FIG. 3 is a schematic block diagram of an embodiment of a distributedstorage processing unit in accordance with the invention;

FIG. 4 is a schematic block diagram of an embodiment of a grid module inaccordance with the invention;

FIG. 5 is a diagram of an example embodiment of error coded data slicecreation in accordance with the invention;

FIG. 6A is a schematic block diagram of an embodiment of a legacycomputing system in accordance with the invention;

FIG. 6B is a graph illustrating an example of a memory status inaccordance with the invention;

FIG. 7A is a schematic block diagram of an embodiment of a hybridcomputing system in accordance with the invention;

FIG. 7B is a schematic block diagram of another embodiment of a hybridcomputing system in accordance with the invention;

FIG. 8A is a schematic block diagram of an embodiment of a dispersedstorage (DS) unit in accordance with the invention;

FIG. 8B is a table illustrating an example of a memory assignment tablein accordance with the invention;

FIG. 9A is a flowchart illustrating an example of allocating memory inaccordance with the invention;

FIG. 9B is a flowchart illustrating another example of allocating memoryin accordance with the invention;

FIG. 10 is a flowchart illustrating an example of rebuilding a dispersedstorage unit in accordance with the invention;

FIG. 11A is a flowchart illustrating an example of commissioning adispersed storage unit in accordance with the invention;

FIG. 11B is another flowchart illustrating another example ofcommissioning a dispersed storage unit in accordance with the invention;

FIG. 12A is a table illustrating an example of a data location table inaccordance with the invention;

FIG. 12B is a flowchart illustrating an example of retrieving data inaccordance with the invention;

FIG. 13 is a flowchart illustrating an example of migrating data inaccordance with the invention;

FIG. 14 is a flowchart illustrating another example of migrating data inaccordance with the invention; and

FIG. 15 is a flowchart illustrating an example of repurposing a memoryin accordance with the invention.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 is a schematic block diagram of a computing system 10 thatincludes one or more of a first type of user devices 12, one or more ofa second type of user devices 14, at least one distributed storage (DS)processing unit 16, at least one DS managing unit 18, at least onestorage integrity processing unit 20, and a distributed storage network(DSN) memory 22 coupled via a network 24. The network 24 may include oneor more wireless and/or wire lined communication systems; one or moreprivate intranet systems and/or public internet systems; and/or one ormore local area networks (LAN) and/or wide area networks (WAN).

The DSN memory 22 includes a plurality of distributed storage (DS) units36 for storing data of the system. Each of the DS units 36 includes aprocessing module and memory and may be located at a geographicallydifferent site than the other DS units (e.g., one in Chicago, one inMilwaukee, etc.). The processing module may be a single processingdevice or a plurality of processing devices. Such a processing devicemay be a microprocessor, micro-controller, digital signal processor,microcomputer, central processing unit, field programmable gate array,programmable logic device, state machine, logic circuitry, analogcircuitry, digital circuitry, and/or any device that manipulates signals(analog and/or digital) based on hard coding of the circuitry and/oroperational instructions. The processing module may have an associatedmemory and/or memory element, which may be a single memory device, aplurality of memory devices, and/or embedded circuitry of the processingmodule. Such a memory device may be a read-only memory, random accessmemory, volatile memory, non-volatile memory, static memory, dynamicmemory, flash memory, cache memory, and/or any device that storesdigital information. Note that if the processing module includes morethan one processing device, the processing devices may be centrallylocated (e.g., directly coupled together via a wired and/or wireless busstructure) or may be distributedly located (e.g., cloud computing viaindirect coupling via a local area network and/or a wide area network).Further note that when the processing module implements one or more ofits functions via a state machine, analog circuitry, digital circuitry,and/or logic circuitry, the memory and/or memory element storing thecorresponding operational instructions may be embedded within, orexternal to, the circuitry comprising the state machine, analogcircuitry, digital circuitry, and/or logic circuitry. Still further notethat, the memory element stores, and the processing module executes,hard coded and/or operational instructions corresponding to at leastsome of the steps and/or functions illustrated in FIGS. 1-15.

Each of the user devices 12-14, the DS processing unit 16, the DSmanaging unit 18, and the storage integrity processing unit 20 may be aportable computing device (e.g., a social networking device, a gamingdevice, a cell phone, a smart phone, a personal digital assistant, adigital music player, a digital video player, a laptop computer, ahandheld computer, a video game controller, and/or any other portabledevice that includes a computing core) and/or a fixed computing device(e.g., a personal computer, a computer server, a cable set-top box, asatellite receiver, a television set, a printer, a fax machine, homeentertainment equipment, a video game console, and/or any type of homeor office computing equipment). Such a portable or fixed computingdevice includes a computing core 26 and one or more interfaces 30, 32,and/or 33. An embodiment of the computing core 26 will be described withreference to FIG. 2.

With respect to the interfaces, each of the interfaces 30, 32, and 33includes software and/or hardware to support one or more communicationlinks via the network 24 and/or directly. For example, interfaces 30support a communication link (wired, wireless, direct, via a LAN, viathe network 24, etc.) between the first type of user device 14 and theDS processing unit 16. As another example, DSN interface 32 supports aplurality of communication links via the network 24 between the DSNmemory 22 and the DS processing unit 16, the first type of user device12, and/or the storage integrity processing unit 20. As yet anotherexample, interface 33 supports a communication link between the DSmanaging unit 18 and any one of the other devices and/or units 12, 14,16, 20, and/or 22 via the network 24.

In general and with respect to data storage, the system 10 supportsthree primary functions: distributed network data storage management,distributed data storage and retrieval, and data storage integrityverification. In accordance with these three primary functions, data canbe distributedly stored in a plurality of physically different locationsand subsequently retrieved in a reliable and secure manner regardless offailures of individual storage devices, failures of network equipment,the duration of storage, the amount of data being stored, attempts athacking the data, etc.

The DS managing unit 18 performs distributed network data storagemanagement functions, which include establishing distributed datastorage parameters, performing network operations, performing networkadministration, and/or performing network maintenance. The DS managingunit 18 establishes the distributed data storage parameters (e.g.,allocation of virtual DSN memory space, distributed storage parameters,security parameters, billing information, user profile information,etc.) for one or more of the user devices 12-14 (e.g., established forindividual devices, established for a user group of devices, establishedfor public access by the user devices, etc.). For example, the DSmanaging unit 18 coordinates the creation of a vault (e.g., a virtualmemory block) within the DSN memory 22 for a user device (for a group ofdevices, or for public access). The DS managing unit 18 also determinesthe distributed data storage parameters for the vault. In particular,the DS managing unit 18 determines a number of slices (e.g., the numberthat a data segment of a data file and/or data block is partitioned intofor distributed storage) and a read threshold value (e.g., the minimumnumber of slices required to reconstruct the data segment).

As another example, the DS managing module 18 creates and stores,locally or within the DSN memory 22, user profile information. The userprofile information includes one or more of authentication information,permissions, and/or the security parameters. The security parameters mayinclude one or more of encryption/decryption scheme, one or moreencryption keys, key generation scheme, and data encoding/decodingscheme.

As yet another example, the DS managing unit 18 creates billinginformation for a particular user, user group, vault access, publicvault access, etc. For instance, the DS managing unit 18 tracks thenumber of times user accesses a private vault and/or public vaults,which can be used to generate a per-access bill. In another instance,the DS managing unit 18 tracks the amount of data stored and/orretrieved by a user device and/or a user group, which can be used togenerate a per-data-amount bill.

The DS managing unit 18 also performs network operations, networkadministration, and/or network maintenance. As at least part ofperforming the network operations and/or administration, the DS managingunit 18 monitors performance of the devices and/or units of the system10 for potential failures, determines the devices and/or unit'sactivation status, determines the devices' and/or units' loading, andany other system level operation that affects the performance level ofthe system 10. For example, the DS managing unit 18 receives andaggregates network management alarms, alerts, errors, statusinformation, performance information, and messages from the devices12-14 and/or the units 16, 20, 22. For example, the DS managing unit 18receives a simple network management protocol (SNMP) message regardingthe status of the DS processing unit 16.

The DS managing unit 18 performs the network maintenance by identifyingequipment within the system 10 that needs replacing, upgrading,repairing, and/or expanding. For example, the DS managing unit 18determines that the DSN memory 22 needs more DS units 36 or that one ormore of the DS units 36 needs updating.

The second primary function (i.e., distributed data storage andretrieval) begins and ends with a user device 12-14. For instance, if asecond type of user device 14 has a data file 38 and/or data block 40 tostore in the DSN memory 22, it send the data file 38 and/or data block40 to the DS processing unit 16 via its interface 30. As will bedescribed in greater detail with reference to FIG. 2, the interface 30functions to mimic a conventional operating system (OS) file systeminterface (e.g., network file system (NFS), flash file system (FFS),disk file system (DFS), file transfer protocol (FTP), web-baseddistributed authoring and versioning (WebDAV), etc.) and/or a blockmemory interface (e.g., small computer system interface (SCSI), internetsmall computer system interface (iSCSI), etc.). In addition, theinterface 30 may attach a user identification code (ID) to the data file38 and/or data block 40.

The DS processing unit 16 receives the data file 38 and/or data block 40via its interface 30 and performs a distributed storage (DS) process 34thereon (e.g., an error coding dispersal storage function). The DSprocessing 34 begins by partitioning the data file 38 and/or data block40 into one or more data segments, which is represented as Y datasegments. For example, the DS processing 34 may partition the data file38 and/or data block 40 into a fixed byte size segment (e.g., 2¹ to2^(n) bytes, where n=>2) or a variable byte size (e.g., change byte sizefrom segment to segment, or from groups of segments to groups ofsegments, etc.).

For each of the Y data segments, the DS processing 34 error encodes(e.g., forward error correction (FEC), information dispersal algorithm,or error correction coding) and slices (or slices then error encodes)the data segment into a plurality of error coded (EC) data slices 42-48,which is represented as X slices per data segment. The number of slices(X) per segment, which corresponds to a number of pillars n, is set inaccordance with the distributed data storage parameters and the errorcoding scheme. For example, if a Reed-Solomon (or other FEC scheme) isused in an n/k system, then a data segment is divided into n slices,where k number of slices is needed to reconstruct the original data(i.e., k is the threshold). As a few specific examples, the n/k factormay be 5/3; 6/4; 8/6; 8/5; 16/10.

For each slice 42-48, the DS processing unit 16 creates a unique slicename and appends it to the corresponding slice 42-48. The slice nameincludes universal DSN memory addressing routing information (e.g.,virtual memory addresses in the DSN memory 22) and user-specificinformation (e.g., user ID, file name, data block identifier, etc.).

The DS processing unit 16 transmits the plurality of EC slices 42-48 toa plurality of DS units 36 of the DSN memory 22 via the DSN interface 32and the network 24. The DSN interface 32 formats each of the slices fortransmission via the network 24.

For example, the DSN interface 32 may utilize an internet protocol(e.g., TCP/IP, etc.) to packetize the slices 42-48 for transmission viathe network 24.

The number of DS units 36 receiving the slices 42-48 is dependent on thedistributed data storage parameters established by the DS managing unit18. For example, the DS managing unit 18 may indicate that each slice isto be stored in a different DS unit 36. As another example, the DSmanaging unit 18 may indicate that like slice numbers of different datasegments are to be stored in the same DS unit 36. For example, the firstslice of each of the data segments is to be stored in a first DS unit36, the second slice of each of the data segments is to be stored in asecond DS unit 36, etc. In this manner, the data is encoded anddistributedly stored at physically diverse locations to improved datastorage integrity and security. Further examples of encoding the datasegments will be provided with reference to one or more of FIGS. 2-15.

Each DS unit 36 that receives a slice 42-48 for storage translates thevirtual DSN memory address of the slice into a local physical addressfor storage. Accordingly, each DS unit 36 maintains a virtual tophysical memory mapping to assist in the storage and retrieval of data.

The first type of user device 12 performs a similar function to storedata in the DSN memory 22 with the exception that it includes the DSprocessing. As such, the device 12 encodes and slices the data fileand/or data block it has to store. The device then transmits the slices11 to the DSN memory via its DSN interface 32 and the network 24.

For a second type of user device 14 to retrieve a data file or datablock from memory, it issues a read command via its interface 30 to theDS processing unit 16. The DS processing unit 16 performs the DSprocessing 34 to identify the DS units 36 storing the slices of the datafile and/or data block based on the read command. The DS processing unit16 may also communicate with the DS managing unit 18 to verify that theuser device 14 is authorized to access the requested data.

Assuming that the user device is authorized to access the requesteddata, the DS processing unit 16 issues slice read commands to at least athreshold number of the DS units 36 storing the requested data (e.g., toat least 10 DS units for a 16/10 error coding scheme). Each of the DSunits 36 receiving the slice read command, verifies the command,accesses its virtual to physical memory mapping, retrieves the requestedslice, or slices, and transmits it to the DS processing unit 16.

Once the DS processing unit 16 has received a read threshold number ofslices for a data segment, it performs an error decoding function andde-slicing to reconstruct the data segment. When Y number of datasegments has been reconstructed, the DS processing unit 16 provides thedata file 38 and/or data block 40 to the user device 14. Note that thefirst type of user device 12 performs a similar process to retrieve adata file and/or data block.

The storage integrity processing unit 20 performs the third primaryfunction of data storage integrity verification. In general, the storageintegrity processing unit 20 periodically retrieves slices 45, and/orslice names, of a data file or data block of a user device to verifythat one or more slices have not been corrupted or lost (e.g., the DSunit failed). The retrieval process mimics the read process previouslydescribed.

If the storage integrity processing unit 20 determines that one or moreslices is corrupted or lost, it rebuilds the corrupted or lost slice(s)in accordance with the error coding scheme. The storage integrityprocessing unit 20 stores the rebuild slice, or slices, in theappropriate DS unit(s) 36 in a manner that mimics the write processpreviously described.

FIG. 2 is a schematic block diagram of an embodiment of a computing core26 that includes a processing module 50, a memory controller 52, mainmemory 54, a video graphics processing unit 55, an input/output (IO)controller 56, a peripheral component interconnect (PCI) interface 58,at least one IO device interface module 62, a read only memory (ROM)basic input output system (BIOS) 64, and one or more memory interfacemodules. The memory interface module(s) includes one or more of auniversal serial bus (USB) interface module 66, a host bus adapter (HBA)interface module 68, a network interface module 70, a flash interfacemodule 72, a hard drive interface module 74, and a DSN interface module76. Note the DSN interface module 76 and/or the network interface module70 may function as the interface 30 of the user device 14 of FIG. 1.Further note that the IO device interface module 62 and/or the memoryinterface modules may be collectively or individually referred to as IOports.

The processing module 50 may be a single processing device or aplurality of processing devices. Such a processing device may be amicroprocessor, micro-controller, digital signal processor,microcomputer, central processing unit, field programmable gate array,programmable logic device, state machine, logic circuitry, analogcircuitry, digital circuitry, and/or any device that manipulates signals(analog and/or digital) based on hard coding of the circuitry and/oroperational instructions. The processing module 50 may have anassociated memory and/or memory element, which may be a single memorydevice, a plurality of memory devices, and/or embedded circuitry of theprocessing module 50. Such a memory device may be a read-only memory,random access memory, volatile memory, non-volatile memory, staticmemory, dynamic memory, flash memory, cache memory, and/or any devicethat stores digital information. Note that if the processing module 50includes more than one processing device, the processing devices may becentrally located (e.g., directly coupled together via a wired and/orwireless bus structure) or may be distributedly located (e.g., cloudcomputing via indirect coupling via a local area network and/or a widearea network). Further note that when the processing module 50implements one or more of its functions via a state machine, analogcircuitry, digital circuitry, and/or logic circuitry, the memory and/ormemory element storing the corresponding operational instructions may beembedded within, or external to, the circuitry comprising the statemachine, analog circuitry, digital circuitry, and/or logic circuitry.Still further note that, the memory element stores, and the processingmodule 50 executes, hard coded and/or operational instructionscorresponding to at least some of the steps and/or functions illustratedin FIGS. 1-15.

FIG. 3 is a schematic block diagram of an embodiment of a dispersedstorage (DS) processing module 34 of user device 12 and/or of the DSprocessing unit 16. The DS processing module 34 includes a gatewaymodule 78, an access module 80, a grid module 82, and a storage module84. The DS processing module 34 may also include an interface 30 and theDSnet interface 32 or the interfaces 68 and/or 70 may be part of user 12or of the DS processing unit 14. The DS processing module 34 may furtherinclude a bypass/feedback path between the storage module 84 to thegateway module 78. Note that the modules 78-84 of the DS processingmodule 34 may be in a single unit or distributed across multiple units.

In an example of storing data, the gateway module 78 receives anincoming data object that includes a user ID field 86, an object namefield 88, and the data field 40 and may also receive correspondinginformation that includes a process identifier (e.g., an internalprocess/application ID), metadata, a file system directory, a blocknumber, a transaction message, a user device identity (ID), a dataobject identifier, a source name, and/or user information. The gatewaymodule 78 authenticates the user associated with the data object byverifying the user ID 86 with the managing unit 18 and/or anotherauthenticating unit.

When the user is authenticated, the gateway module 78 obtains userinformation from the management unit 18, the user device, and/or theother authenticating unit. The user information includes a vaultidentifier, operational parameters, and user attributes (e.g., userdata, billing information, etc.). A vault identifier identifies a vault,which is a virtual memory space that maps to a set of DS storage units36. For example, vault 1 (i.e., user 1's DSN memory space) includeseight DS storage units (X=8 wide) and vault 2 (i.e., user 2's DSN memoryspace) includes sixteen DS storage units (X=16 wide). The operationalparameters may include an error coding algorithm, the width n (number ofpillars X or slices per segment for this vault), a read threshold T, awrite threshold, an encryption algorithm, a slicing parameter, acompression algorithm, an integrity check method, caching settings,parallelism settings, and/or other parameters that may be used to accessthe DSN memory layer.

The gateway module 78 uses the user information to assign a source name35 to the data. For instance, the gateway module 60 determines thesource name 35 of the data object 40 based on the vault identifier andthe data object. For example, the source name may contain a fileidentifier (ID), a vault generation number, a reserved field, and avault identifier (ID). As another example, the gateway module 78 maygenerate the file ID based on a hash function of the data object 40.Note that the gateway module 78 may also perform message conversion,protocol conversion, electrical conversion, optical conversion, accesscontrol, user identification, user information retrieval, trafficmonitoring, statistics generation, configuration, management, and/orsource name determination.

The access module 80 receives the data object 40 and creates a series ofdata segments 1 through Y 90-92 in accordance with a data storageprotocol (e.g., file storage system, a block storage system, and/or anaggregated block storage system). The number of segments Y may be chosenor randomly assigned based on a selected segment size and the size ofthe data object. For example, if the number of segments is chosen to bea fixed number, then the size of the segments varies as a function ofthe size of the data object. For instance, if the data object is animage file of 4,194,304 eight bit bytes (e.g., 33,554,432 bits) and thenumber of segments Y=131,072, then each segment is 256 bits or 32 bytes.As another example, if segment sized is fixed, then the number ofsegments Y varies based on the size of data object. For instance, if thedata object is an image file of 4,194,304 bytes and the fixed size ofeach segment is 4,096 bytes, the then number of segments Y=1,024. Notethat each segment is associated with the same source name.

The grid module 82 receives the data segments and may manipulate (e.g.,compression, encryption, cyclic redundancy check (CRC), etc.) each ofthe data segments before performing an error coding function of theerror coding dispersal storage function to produce a pre-manipulateddata segment. After manipulating a data segment, if applicable, the gridmodule 82 error encodes (e.g., Reed-Solomon, Convolution encoding,Trellis encoding, etc.) the data segment or manipulated data segmentinto X error coded data slices 42-44.

The value X, or the number of pillars (e.g., X=16), is chosen as aparameter of the error coding dispersal storage function. Otherparameters of the error coding dispersal function include a readthreshold T, a write threshold W, etc. The read threshold (e.g., T=10,when X=16) corresponds to the minimum number of error-free error codeddata slices required to reconstruct the data segment. In other words,the DS processing module 34 can compensate for X−T (e.g., 16−10=6)missing error coded data slices per data segment. The write threshold Wcorresponds to a minimum number of DS storage units that acknowledgeproper storage of their respective data slices before the DS processingmodule indicates proper storage of the encoded data segment. Note thatthe write threshold is greater than or equal to the read threshold for agiven number of pillars (X).

For each data slice of a data segment, the grid module 82 generates aunique slice name 37 and attaches it thereto. The slice name 37 includesa universal routing information field and a vault specific field and maybe 48 bytes (e.g., 24 bytes for each of the universal routinginformation field and the vault specific field). As illustrated, theuniversal routing information field includes a slice index, a vault ID,a vault generation, and a reserved field. The slice index is based onthe pillar number and the vault ID and, as such, is unique for eachpillar (e.g., slices of the same pillar for the same vault for anysegment will share the same slice index). The vault specific fieldincludes a data name, which includes a file ID and a segment number(e.g., a sequential numbering of data segments 1-Y of a simple dataobject or a data block number).

Prior to outputting the error coded data slices of a data segment, thegrid module may perform post-slice manipulation on the slices. Ifenabled, the manipulation includes slice level compression, encryption,CRC, addressing, tagging, and/or other manipulation to improve theeffectiveness of the computing system.

When the error coded data slices of a data segment are ready to beoutputted, the grid module 82 determines which of the DS storage units36 will store the EC data slices based on a dispersed storage memorymapping associated with the user's vault and/or DS storage unitattributes. The DS storage unit attributes may include availability,self-selection, performance history, link speed, link latency,ownership, available DSN memory, domain, cost, a prioritization scheme,a centralized selection message from another source, a lookup table,data ownership, and/or any other factor to optimize the operation of thecomputing system. Note that the number of DS storage units 36 is equalto or greater than the number of pillars (e.g., X) so that no more thanone error coded data slice of the same data segment is stored on thesame DS storage unit 36. Further note that EC data slices of the samepillar number but of different segments (e.g., EC data slice 1 of datasegment 1 and EC data slice 1 of data segment 2) may be stored on thesame or different DS storage units 36.

The storage module 84 performs an integrity check on the outboundencoded data slices and, when successful, identifies a plurality of DSstorage units based on information provided by the grid module 82. Thestorage module 84 then outputs the encoded data slices 1 through X ofeach segment 1 through Y to the DS storage units 36. Each of the DSstorage units 36 stores its EC data slice(s) and maintains a localvirtual DSN address to physical location table to convert the virtualDSN address of the EC data slice(s) into physical storage addresses.

In an example of a read operation, the user device 12 and/or 14 sends aread request to the DS processing unit 14, which authenticates therequest. When the request is authentic, the DS processing unit 14 sendsa read message to each of the DS storage units 36 storing slices of thedata object being read. The slices are received via the DSnet interface32 and processed by the storage module 84, which performs a parity checkand provides the slices to the grid module 82 when the parity check wassuccessful. The grid module 82 decodes the slices in accordance with theerror coding dispersal storage function to reconstruct the data segment.The access module 80 reconstructs the data object from the data segmentsand the gateway module 78 formats the data object for transmission tothe user device.

FIG. 4 is a schematic block diagram of an embodiment of a grid module 82that includes a control unit 73, a pre-slice manipulator 75, an encoder77, a slicer 79, a post-slice manipulator 81, a pre-slice de-manipulator83, a decoder 85, a de-slicer 87, and/or a post-slice de-manipulator 89.Note that the control unit 73 may be partially or completely external tothe grid module 82. For example, the control unit 73 may be part of thecomputing core at a remote location, part of a user device, part of theDS managing unit 18, or distributed amongst one or more DS storageunits.

In an example of write operation, the pre-slice manipulator 75 receivesa data segment 90-92 and a write instruction from an authorized userdevice. The pre-slice manipulator 75 determines if pre-manipulation ofthe data segment 90-92 is required and, if so, what type. The pre-slicemanipulator 75 may make the determination independently or based oninstructions from the control unit 73, where the determination is basedon a computing system-wide predetermination, a table lookup, vaultparameters associated with the user identification, the type of data,security requirements, available DSN memory, performance requirements,and/or other metadata.

Once a positive determination is made, the pre-slice manipulator 75manipulates the data segment 90-92 in accordance with the type ofmanipulation. For example, the type of manipulation may be compression(e.g., Lempel-Ziv-Welch, Huffman, Golomb, fractal, wavelet, etc.),signatures (e.g., Digital Signature Algorithm (DSA), Elliptic Curve DSA,Secure Hash Algorithm, etc.), watermarking, tagging, encryption (e.g.,Data Encryption Standard, Advanced Encryption Standard, etc.), addingmetadata (e.g., time/date stamping, user information, file type, etc.),cyclic redundancy check (e.g., CRC32), and/or other data manipulationsto produce the pre-manipulated data segment.

The encoder 77 encodes the pre-manipulated data segment 92 using aforward error correction (FEC) encoder (and/or other type of erasurecoding and/or error coding) to produce an encoded data segment 94. Theencoder 77 determines which forward error correction algorithm to usebased on a predetermination associated with the user's vault, a timebased algorithm, user direction, DS managing unit direction, controlunit direction, as a function of the data type, as a function of thedata segment 92 metadata, and/or any other factor to determine algorithmtype. The forward error correction algorithm may be Golay,Multidimensional parity, Reed-Solomon, Hamming, Bose Ray ChauduriHocquenghem (BCH), Cauchy-Reed-Solomon, or any other FEC encoder. Notethat the encoder 77 may use a different encoding algorithm for each datasegment 92, the same encoding algorithm for the data segments 92 of adata object, or a combination thereof.

The encoded data segment 94 is of greater size than the data segment 92by the overhead rate of the encoding algorithm by a factor of X/T, whereX is the width or number of slices, and T is the read threshold. In thisregard, the corresponding decoding process can accommodate at most X-Tmissing EC data slices and still recreate the data segment 92. Forexample, if X=16 and T=10, then the data segment 92 will be recoverableas long as 10 or more EC data slices per segment are not corrupted.

The slicer 79 transforms the encoded data segment 94 into EC data slicesin accordance with the slicing parameter from the vault for this userand/or data segment 92.

For example, if the slicing parameter is X=16, then the slicer 79 sliceseach encoded data segment 94 into 16 encoded slices.

The post-slice manipulator 81 performs, if enabled, post-manipulation onthe encoded slices to produce the EC data slices. If enabled, thepost-slice manipulator 81 determines the type of post-manipulation,which may be based on a computing system-wide predetermination,parameters in the vault for this user, a table lookup, the useridentification, the type of data, security requirements, available DSNmemory, performance requirements, control unit directed, and/or othermetadata. Note that the type of post-slice manipulation may includeslice level compression, signatures, encryption, CRC, addressing,watermarking, tagging, adding metadata, and/or other manipulation toimprove the effectiveness of the computing system.

In an example of a read operation, the post-slice de-manipulator 89receives at least a read threshold number of EC data slices and performsthe inverse function of the post-slice manipulator 81 to produce aplurality of encoded slices. The de-slicer 87 de-slices the encodedslices to produce an encoded data segment 94. The decoder 85 performsthe inverse function of the encoder 77 to recapture the data segment90-92. The pre-slice de-manipulator 83 performs the inverse function ofthe pre-slice manipulator 75 to recapture the data segment 90-92.

FIG. 5 is a diagram of an example of slicing an encoded data segment 94by the slicer 79. In this example, the encoded data segment 94 includesthirty-two bits, but may include more or less bits. The slicer 79disperses the bits of the encoded data segment 94 across the EC dataslices in a pattern as shown. As such, each EC data slice does notinclude consecutive bits of the data segment 94 reducing the impact ofconsecutive bit failures on data recovery. For example, if EC data slice2 (which includes bits 1, 5, 9, 13, 17, 25, and 29) is unavailable(e.g., lost, inaccessible, or corrupted), the data segment can bereconstructed from the other EC data slices (e.g., 1, 3 and 4 for a readthreshold of 3 and a width of 4).

FIG. 6A is a schematic block diagram of an embodiment of a legacycomputing system. The system includes a plurality of memories 102-106, aplurality of memory units 108-110, a memory bus 118, and a computingcore 26. The memory units 108-110 may include a plurality of memories112-116. The memories 102-106 and 112-116 may include one or more of amagnetic hard drives, a solid state memory, a tape drive, and opticalmemory, or any other type of memory technology to store and retrievedata. The memories 102-106 and 112-116 may have varying capacities. Forexample, memory 102 has a 500 gigabyte (GB) capacity, memory 104 as a 1terabyte (TB) capacity, and memory 106 as a 2 TB capacity. The memories102-106 and 110-116 may be implemented with different models ofdifferent manufacturers.

The memories 102-106 and 112-116 are operably coupled to the computingcore 26 via the memory bus 118 to facilitate transfer of data 120-124and data 126-132. For example, memory 104 communicates data 122 with thecomputing core 26. The computing core 26 may store replicated copies ofthe same data in two or more of the memories. For example, computingcore 26 stores a first copy of data in memory 106 and a second copy ofthe data in memory 114. As another example, computing core 26 stores thefirst copy of data in memory 112 and a second copy of the data in memory116 when the data is to be replicated within a single memory unit 108.

A typical memory of the memories 102-106 and 112-116 may fail from timeto time as the memory ages beyond a usable memory life time period.Memory costs of the computing system include memory replacement costsand memory cost over the usable memory life. The memory cost over theusable memory life includes the memory cost divided by the usable memorylife time period. Lowering the memory cost lowers the memory costs ofthe computing system. Extending the usable memory life time periodlowers the memory costs of the computing system. Replacing the memoryimpacts cost of the legacy computing system based on a memoryreplacement cost and a memory disposal cost.

FIG. 6B is a graph illustrating an example of a memory status. The graphdepicts usable memory (e.g., capacity) over time for a memory device(e.g., a magnetic hard disk drive). Usability is stratified into fourcategories including a level 1 usability, a level 2 usability, a level 3usability, and an unusable level. Note that a maximum amount of usablememory is available during the level 1 time period. A degradation ofusable memory occurs over time as the memory device ages. For example, ahard failure abruptly changes the usable memory level from a usablelevel (e.g., level 1) to the unusable level. As another example, softfailures may gradually change the usable level from level 1 to level 2to level 3 to the unusable level when the usable memory is below athreshold.

The memories 102-106 and 112-116 may be of different ages with respectto initial use such that some memories may fail sooner than othermemories. Each memory may follow the memory status curve of the graph indifferent ways. Analyzing or predicting a memory status of a memory mayprovide an improvement in memory utilization by avoiding use of a memorythat is not favorable for storing data based on an associated storagerequirement of the data. A catastrophic loss of data may occur if onlyone copy of data is stored on a memory that follows the soft failurecurve such that eventually the data is no longer retrievable. Acatastrophic loss of data may not occur when one pillar of encoded dataslices, produced using an error coding dispersal storage function, isstored on a memory that follows the soft failure curve such thateventually the pillars is no longer retrievable. In such a scenario, thedata may be still be retrievable when a threshold number of encoded dataslices from other memories can be retrieved. A single memory failure maybe much less likely to cause a catastrophic loss of data when the memoryis utilized in a dispersed storage network. An improvement to theoverall usable memory life may be provided by migrating a memory from alegacy memory system to a dispersed storage network. The method ofmigrating a memory from the legacy memory system to the dispersedstorage network is discussed in greater detail with reference to FIGS.7A-15.

FIGS. 7A-7B illustrates a memory migration scenario illustrating howdata contained in a memory of a legacy computing system may be migratedto a dispersed storage network (DSN) and how the memory may bephysically repurposed in the dispersed storage to store encoded dataslices rather than data (e.g., whole data objects). A reliability andavailability improvement of the data may be provided in such a migrationscenario since the data is stored as encoded data slices and failure ofthe memory may not impact data availability. An improvement in theuseful life of the memory may be provided in the migration scenariosince a failure of a memory in the DSN may not impact data availabilityto the degree that a failure of the memory when utilized in the legacycomputing system. The method of operation of migrating data and memoriesfrom a legacy computing system to a DSN is discussed in greater detailwith reference to FIGS. 8-15.

FIG. 7A is a schematic block diagram of an embodiment of a hybridcomputing system. The hybrid computing system includes a legacycomputing system operably coupled to a dispersed storage network (DSN).The system includes a plurality of memories 102-106, a plurality ofmemory units 108-110, a memory bus 118, a computing core 26, a dispersedstorage (DS) processing unit 16, and a DSN memory 22. As illustrated,the DSN memory 22 includes a plurality of DS units 36. Memory unit 108includes a plurality of memories 112-116. The computing core 26interoperates with the DS processing unit 16 as described below.

The DS processing unit 16 encodes data utilizing an error codingdispersal storage function to produce encoded data slices 11. The DSprocessing unit 16 outputs the encoded data slices 11 to the DSN memory22 for storage. The DS processing unit 16 retrieves encoded data slices11 from the DSN memory and decodes the encoded data slices 11 utilizingthe error coding dispersal storage function to reproduce the data. In anexample of operation, the computing core 26 retrieves data 122 frommemory 104 when memory device 104 has an expired usable memory life withrespect to a legacy storage protocol utilized by the legacy computingsystem. The computing core 26 sends the data 122 to the DS processingunit 16. The DS processing unit 16 encodes the data 122 utilizing theerror coding dispersal storage function to produce encoded data slicesof data 122. The DS processing unit 16 sends the encoded data slices 11to the DSN memory 22 for storage in a plurality of DS units 36. Notethat the computing core 26 may retrieve the data 122 either from memory104 or from the DS processing unit 16. The computing core 26 retrievesthe data 122 from the DS processing unit 16 when the memory 104 isremoved from the legacy computing system. The method of retrieval ofdata 122 is discussed in greater detail with reference to FIG. 7B.

FIG. 7B is another schematic block diagram of another embodiment of ahybrid computing system. The hybrid computing system includes a legacycomputing system and a dispersed storage network (DSN). The systemincludes a plurality of memories 102-106, a plurality of memory units108-110, a memory bus 118, a computing core 26, a dispersed storage (DS)processing unit 16, and a DSN memory 22. The memory 104 is physicallymoved from the legacy computing system (e.g., disconnected from thememory bus 118 as shown in FIG. 7A) to the DSN when the memory 104 hasan expired usable memory life and data 122 has been extracted from thememory 104. The memory 104 is utilized as a DS unit 36 as part of theDSN memory 22 when memory 104 is moved to the DSN. For example, thememory 104 receives encoded data slices for storage from the DSprocessing unit 16. As another example, the memory 104 outputs encodeddata slices to the DS processing unit 16 in response to a retrievalrequest.

In an example of operation, the computing core 26 sends a retrievalrequest to the DS processing unit 16 for data 122. The DS processingunit 16 retrieves encoded data slices 11 from the DSN memory 22. The DSprocessing unit 16 decodes the encoded data slices 11 utilizing an errorcoding dispersal storage function to reproduce the data 122. The DSprocessing unit 16 outputs the data 122 to the computing core 26. In aninstance, at least some of the encoded data slices 11 (e.g., of data122) are stored within the memory 104. In another instance, none of theencoded data slices 11 (e.g., of data 122) are stored within the memory104.

FIG. 8A is a schematic block diagram of an embodiment of a dispersedstorage (DS) unit 36. The DS unit 36 includes a plurality of memories1_1 to M_N, and a memory control module 146. The DS unit 36 may beimplemented with any number of memories. The memories 1_1 to M_N may bephysically repurposed to the DS unit from a legacy computing system. Thememory control module 146 may be implemented utilizing a computing core26. The memory control module 146 is operably coupled to each of theplurality of memories 1_1 to M_N. The memory control module 146functions include one or more of controlling the memories, storing data,retrieving data, deleting data, listing data, configuring memories,allocating memories, determining status, storing metadata, storingencryption keys, storing memory device access information, and managingmemories. The memory control module 146 interfaces to the network 24 tofacilitate communication of control information 148 and slices 11 with adispersed storage network (DSN).

In an example of operation, the memory control module 146 receivesslices 11 via the network 24. The memory control module 146 selects oneor more of the memories 1_1 to M_N to produce selected memories to storethe slices 11 based on one or more of a vault identifier, a useridentifier, a data identifier, a current allocation of memories tovaults, memory status, a memory age indicator, an error message, amemory performance history record, and a storage requirement. The memorycontrol module 146 stores the slices in the selected memories.

As another example of operation, the memory control module 146determines a memory status of a memory based on one or more of a query,a test, a performance record, an availability record, a reliabilityrecord, an error message, a memory age indicator, a usable memory lifeindicator, a set of usable memory life thresholds, a previous memorystatus, a message, a usable portion of the memory indicator, an unusableportion of the memory indicator, and a command. For instance, the memorycontrol module 146 determines the memory status based on verifyingoperation via a test of one or more portions of the memory. The methodof operation of the memory control module 146 is discussed in greaterdetail with reference to FIG. 8B-15.

FIG. 8B is a table illustrating an example of a memory assignment table150. The memory assignment table 150 includes a memory identifier (ID)field 152, an allocation field 154, and a memory status field 156. Thememory ID field 152 includes memory ID entries that list an identifierassociated with a particular memory such that substantially all memoriesof a dispersed storage (DS) unit are listed within the memory assignmenttable 150 (e.g., memories 1_1 to M_N).

The allocation field 154 includes allocation entries that indicatewhether a memory of an associated memory ID is unallocated forutilization or allocated for utilization to a vault of a dispersedstorage network (DSN). For example, memories 1_1, 2_2, M_2, and M_N areunallocated, memory 1_2 is allocated to vault 320, memory 1_N isallocated to vault 59B, memories 2_1 and 2_N are allocated to vault 10A,and memory M_1 is allocated to vault 457. Two or more memories may beallocated to the same vault when the two or more memories are utilizedto store slices of two or more pillars. Two or more memories may beallocated to the same vault when the two or more memories are utilizedto store sub-slices of a slice received via the network 24.

The memory status field 156 includes memory status entries that indicatea memory status of an associated memory. For example, memory 1_1 has alevel 3 memory status, memory 1_2 as a level 1 memory status, memory 1_Nhas a level 2 memory status, memory 2_1 has the level 1 memory status,memory 2_2 as an unusable memory status, memory 2_N has the level 2memory status, memory M_1 as the level 1 memory status, memory M_2 asthe unusable memory status, and memory M_N has the level 1 memorystatus.

FIG. 9A is a flowchart illustrating an example of allocating memory. Themethod begins with step 160 where a processing module (e.g., of a memorycontrol module) determines a memory usability indication for a set ofmemory devices, wherein the set of memory devices stores data as firstdispersed storage error coded data using first dispersed storage errorcoding parameters. In an embodiment, a memory device of the set ofmemory devices has been reprovisioned from a legacy storage protocol toan error coding dispersed storage protocol. The processing moduledetermines the memory usability indication by at least one of querying amemory usability level table, testing the set of memory devices,determining a memory assignment, receiving an error message, retrievingan error message history, receiving a message, and receiving a command.

The method continues at step 162 where the processing module comparesthe memory usability indication to a memory usability level threshold.The method branches to step 166 when the comparison is unfavorable. Themethod continues to step 164 when the comparison is favorable. Themethod ends at step 164.

The method continues at step 166 where the processing module adds one ormore memory devices to the set of memory devices to produce an updatedset of memory devices when the memory usability indication comparesunfavorably to the memory usability level threshold. For example, theprocessing module adds two memory devices that were not part of the setof memory devices to set a memory devices to produce the updated set ofmemory devices. Alternatively, the processing module compares the memoryusability indication to a plurality of memory usability levelthresholds, wherein the plurality of memory usability level thresholdsincludes the memory usability level threshold, based on the comparingthe memory usability indication to the plurality of memory usabilitylevel thresholds, determines a number of memory devices to add to theset of memory devices to produce a determined number of memory devices,and adds, as the one or more memory devices, the determined number ofmemory devices to the set of memory devices. The processing module maycompare the memory usability indication to the plurality of memoryusability level thresholds by one or more of comparing a usable memorylife of the set of memory devices to the plurality of memory usabilitylevel thresholds, comparing storage requirements associated with thedata to the plurality of memory usability level thresholds, querying amemory usability level table, testing the set of memory devices,determining a memory assignment, receiving an error message, retrievingan error message history, receiving a message, and receiving a command.

The method continues at step 168 where the processing module stores thedata as second dispersed storage error coded data using second dispersedstorage error coding parameters in the updated set of memory devices.Storing the data as second dispersed storage error coded data includesselecting, based on the comparing the memory usability indication to thememory usability level threshold, the second dispersed storage errorcoding parameters to have a decode threshold substantially equal to adecode threshold of the first dispersed storage error coding parametersand to have a larger pillar width than a pillar width of the firstdispersed storage error coding parameters. For example, the processingmodule selects the second dispersed storage error coding parameters toinclude a pillar width of 18 and a decode threshold of 10 when the firstdispersed storage error coding parameters includes a pillar width of 16and a decode threshold of 10. Next, the processing module obtains thedata and dispersed storage error encodes the data utilizing the seconddispersed storage error coding parameters to produce the data as seconddispersed storage error coded data (e.g., a plurality of sets of encodeddata slices). The processing module stores encoded data slicesassociated with pillars 17 and 18 in memory devices added to the set ofmemory devices. The processing module updates a virtual dispersedstorage network (DSN) address to physical location table to indicatewhich memory devices are storing which pillars.

Alternatively, the storing the data as second dispersed storage errorcoded data includes selecting, based on the comparing the memoryusability indication to the memory usability level threshold, the seconddispersed storage error coding parameters to have a smaller decodethreshold than a decode threshold of the first dispersed storage errorcoding parameters and to have a pillar width that is substantially equalto or greater than a pillar width of the first dispersed storage errorcoding parameters. For example, the processing module selects the seconddispersed storage error coding parameters to include a pillar width of18 and a decode threshold of 8 when the first dispersed storage errorcoding parameters includes a pillar width of 16 and a decode thresholdof 10. Next, the processing module obtains the data and dispersedstorage error encodes the data utilizing the second dispersed storageerror coding parameters to produce the data as second dispersed storageerror coded data. The processing module stores the second dispersedstorage error coded data in the updated set of memory devices. Forinstance, each pillar of 18 pillars is stored in a unique memory of theupdated set of memory devices that includes the memory devices added tothe set of memory devices.

Alternatively, the processing module compares the memory usabilityindication to a plurality of memory usability level thresholds, whereinthe plurality of memory usability level thresholds includes the memoryusability level threshold and based on the comparing the memoryusability indication to the plurality of memory usability levelthresholds, selects the second dispersed storage error coding parametersfrom a plurality of dispersed storage error coding parameters.

FIG. 9B is a flowchart illustrating another example of allocatingmemory, which include similar steps to FIG. 9A. The method begins withsteps 160-162 of FIG. 9A where a processing module (e.g., of a memorycontrol module) determines a memory usability indication for a set ofmemory devices, wherein the set of memory devices stores data as firstdispersed storage error coded data using first dispersed storage errorcoding parameters and compares the memory usability indication to amemory usability level threshold. The method branches to step 170 whenthe comparison is unfavorable. The method continues to step 164 of FIG.9A when the comparison is favorable. The method ends at step 164 of FIG.9A.

The method continues at step 170 where the processing module determines,based on the comparison of the memory usability indication and thememory usability level threshold at least one of whether to add one ormore memory devices to the set of memory devices and whether to changethe first dispersed storage error coding parameters. Alternatively, theprocessing module makes a default determination to add one or morememory devices to the set of memory devices and to change the firstdispersed storage error coding parameters. For example, processingmodule determines to add one or more memory devices when the comparisonindicates that a significant reliability improvement is required. Asanother example, the processing module determines to only change thefirst dispersed storage error coding parameters when the comparisonindicates that a modest reliability improvement is required. The methodcontinues at step 172 where the processing module determines whether toadd one or more member devices to the set of memory devices based on thedetermining whether to add one or more member devices to the centermember devices and whether to change first dispersed storage errorcoding parameters.

The method branches to step 174 when the processing module determinesnot to change the first dispersed storage error coding parameterswithout adding memory. The method continues to step 166 of FIG. 9A whenthe processing module determines to add memory. The method continueswith steps 166-168 of FIG. 9A where the processing module adds one ormore memory devices to the set of memory devices to produce an updatedset of memory devices when the determination is to add the one or morememory devices and the processing module stores the data as seconddispersed storage error coded data using second dispersed storage errorcoding parameters in the updated set of memory devices.

The method continues at step 174 where the processing module selects thesecond dispersed storage error coding parameters based on the comparisonof the memory usability indication to the memory usability levelthreshold. The method continues at step 176 where the processing modulestores the data as the second dispersed storage error coded data usingsecond dispersed storage error coding parameters in at least one of theset of memory devices and the updated set of memory devices.

FIG. 10 is a flowchart illustrating an example of rebuilding a dispersedstorage unit. The method begins with step 180 where a processing module(e.g., of a memory control module) determines a current memory status ofmemories associated with a dispersed storage (DS) unit. Thedetermination may be based on one or more of a lookup in a memoryassignment table, a test, a query, and retrieving a historicalperformance record. The method continues at step 182 where theprocessing module determines whether the memory status comparesfavorably to a status threshold. For example, the processing moduledetermines that the memory status compares favorably to the statusthreshold when a number of errors of a memory is less than an errorthreshold of the status threshold. The method repeats back to step 180when the processing module determines that the memory status doescompare favorably to the status threshold. The method continues to step184 when the processing module determines that the memory status doesnot compare favorably to the status threshold (e.g., too many errors).

The method continues at step 184 where the processing moduledecommissions the DS unit. The decommissioning may include one or moreof shutting down the DS unit, sending an error message to a DS managingunit, and changing a status indicator for the DS unit. The methodcontinues at step 186 where the processing module detectsre-commissioning of the DS unit. The detection may be based on one ormore of a message, a command, a request, an error message, and a DS unitidentifier. The DS unit may have been repaired such that one or morefailing memories may have been replaced with replacement memories whenthe DS unit is re-commissioned.

The method continues at step 188 where the processing module determinesreplacement memories and original memories (e.g., those not replaced).The determination may be based on one or more of a memory query, avirtual dispersed storage network (DSN) address to physical locationtable lookup, a list, and detecting encoded data slices. For example,the processing module determines that a memory is a replacement memorywhen there are no previously stored encoded data slices stored on thememory. As another example, the processing module determines that amemory is an original memory when there are pervasive stored encodeddata slices stored on the memory.

The method continues at step 190 where the processing module rebuildsall slices associated with the replacement memories. Such slices aremissing for those slices stored previous to the decommissioning of theDS unit in addition to the slices that may have been stored between thetime that the DS unit was decommissioned and subsequentlyre-commissioned. For example, the processing module decodes a thresholdnumber of retrieved slices from other memories to produce data,dispersed error encodes the data to produce encoded data slicescorresponding to the pillars of the replacement memories, and stores theencoded data slices in the corresponding replacement memories of the DSunit.

The method continues at step 192 where the processing module rebuildsall slices associated with the original memories. Such slices aremissing for those slices stored between the time that the DS unit wasdecommissioned and subsequently re-commissioned. For example, theprocessing module decodes a threshold number of retrieved slices fromother memories to produce data, dispersed error encodes the data toproduce encoded data slices corresponding to the pillars of the originalmemories, and stores the encoded data slices in the correspondingoriginal memories of the DS unit. In addition, the processing module maydelete slices that were deleted between the time that the DS unit wasdecommissioned and the time when the DS unit was re-commissioned.

FIG. 11A is a flowchart illustrating an example of commissioning adispersed storage unit, which includes similar steps to FIG. 10. Themethod begins with steps 180-182 of FIG. 10 where a processing module(e.g., of a memory control module) determines a current memory status ofmemories associated with a dispersed storage (DS) unit and whether thememory status compares favorably to a status threshold. The methodrepeats back to step 180 of FIG. 10 when the processing moduledetermines that the memory status does compare favorably to the statusthreshold. The method continues to step 184 of FIG. 10 when theprocessing module determines that the memory status does not comparefavorably to the status threshold (e.g., too many errors). The methodcontinues at step 184 of FIG. 10 where the processing moduledecommissions the DS unit.

The method continues at step 194 where the processing module detectscommissioning of a second DS unit. The detection may be based on one ormore of a message, a command, a request, an error message, and a DS unitidentifier. The DS unit may have been permanently retired and replacedwith the second DS unit.

The method continues with step 196 where the processing module rebuildsall slices of all memories associated with the second DS unit. Suchslices are missing for those slices stored previous to thedecommissioning of the DS unit in addition to the slices that may havebeen stored (e.g., to memories of other pillars) between the time thatthe DS unit was decommissioned and the second DS unit was commissioned.For example, the processing module decodes a threshold number ofretrieved slices from other memories to produce data, dispersed errorencodes the data to produce encoded data slices corresponding to thepillars of the replacement memories, and stores the encoded data slicesin the corresponding memories of the second DS unit.

FIG. 11B is another flowchart illustrating another example ofcommissioning a dispersed storage unit, which includes similar steps toFIGS. 10 and 11A. The method begins with steps 180-182 of FIG. 10 wherea processing module (e.g., of a memory control module) determines acurrent memory status of memories associated with a dispersed storage(DS) unit and determines whether the memory status compares favorably toa status threshold. The method repeats back to step 180 of FIG. 10 whenthe processing module determines that the memory status does comparefavorably to the status threshold. The method continues to step 198 whenthe processing module determines that the memory status does not comparefavorably to the status threshold (e.g., too many errors).

The method continues at step 198 where the processing module cachesslices of the DS unit (e.g., into other DS units). The method continueswith step 184 of FIG. 10 where the processing module decommissions theDS unit. The method continues with step 194 of FIG. 11A where theprocessing module detects commissioning of a second DS unit. The DS unitmay have been permanently retired and replaced with the second DS unit.

The method continues at step 200 where the processing module retrievesthe cached slices. Pour example, the processing module sends a retrieveslice command to one or more memories of one or more other DS units andreceives the cached slices in response. The method continues at step 202where the processing module stores the cached slices in the second DSunit. The method continues at step 204 where the processing modulerebuilds missing slices of the second DS unit. Such slices are missingfor those slices stored between the time that the DS unit wasdecommissioned and the second DS unit was commissioned. For example, theprocessing module decodes a decodes threshold number of retrieved slicesfrom other memories to produce data, dispersed error encodes the data toproduce encoded data slices corresponding to the pillars of thereplacement memories, and stores the encoded data slices in thecorresponding memories of the second DS unit.

FIG. 12A is a table illustrating an example of a data location table 206that includes a data identifier (ID) field 208, a legacy memory addressfield 210, and a dispersed storage network (DSN) address field 212. Thedata location table 206 may be utilized to identify storage locations ofdata wherein the data may be stored in one or more of a legacy memoryand a dispersed storage network. The data ID field 208 includes one ormore data ID entries, wherein each data ID entry includes at least oneof an object identifier, a filename, a source name, a slice name, andany other identifier of a data file. For example, data 122 represents aword processing data file. The legacy memory address field 210 includesone or more legacy memory address entries corresponding to one or moredata IDs, wherein the legacy memory address includes a memory identifierof a memory of a legacy computing system (e.g., not a dispersed storagenetwork). For example, legacy memory address 104 is utilized to storedata identified by data 122.

The DSN address field 212 includes one or more DSN address entriescorresponding to one or more data IDs, wherein the DSN address includesat least one of a source name and a slice name. For example, the dataidentified by data 122 is stored in the DSN system at DSN address 1AC.Data may be stored in one or both of the legacy memory and the DSNsystem. For example, data 132 is stored in legacy memory 110 but is notstored in the DSN system. As another example, data 134 is not stored inthe legacy memory but is stored in the DSN system at DSN address 2D5. Asyet another example, data 122 is stored in legacy memory 104 and in theDSN system at DSN address 1AC. The method of utilization of the datalocation table is discussed in greater detail with reference to FIG.12B.

FIG. 12B is a flowchart illustrating an example of retrieving data. Themethod begins with step 214 where a processing module (e.g. of adispersed storage (DS) unit) receives a retrieval request for a file.The request may include one or more of a data identifier (ID), a legacymemory address, a dispersed storage network (DSN) address, a datalocation table, a migration indicator, and a requesting entity ID.

The method continues at step 216 where the processing module determineswhether the file is being migrated from a legacy storage system to adispersed error coding storage system. The determination may be based ondetermining whether that the file is stored in a legacy format in thelegacy storage system and is stored as a plurality of sets of encodeddata slices in the dispersed error coding storage system. For example,the processing module accesses a data location table and determines thatthe file is being migrated when the file is stored in the legacy formatand is stored as the plurality of sets of encoded data slices. Asanother example, the processing module determines that the file is beingmigrated when the migration indicator indicates that the file is beingmigrated. The method branches to step 221 the processing moduledetermines that the file is being migrated. The method continues to stepto step 218 when the processing module determines that the file is notbeing migrated. The method continues at step 218 where the processingmodule obtains the file. The obtaining includes one or more ofperforming the data location table lookup to determine a locationcorresponding to the file, accessing the file from the legacy storagesystem, and accessing the file from the dispersed error coding storagesystem.

The method continues at step 220 where the processing module determinesa retrieval option for the file when the file is being migrated from thelegacy storage system to the dispersed error coding storage system. Theretrieval option includes at least one of retrieve the legacy formatfrom the legacy storage system only, retrieve the plurality of sets ofencoded data slices from the dispersed error coding storage system only,and retrieve both the legacy format from the legacy storage system andthe plurality of sets of encoded data slices from the dispersed errorcoding storage system and outputting, based on first available, thelegacy format or the plurality of sets of encoded data slices. Theprocessing module determines the retrieval option for the file based onat least one of a retrieval latency requirement, a retrieval reliabilityrequirement, a memory availability requirement, and a maximum bandwidthutilization requirement. For example, the processing module determinesthe retrieval option to include retrieving the legacy format when theretrieval latency requirement indicates a low latency is required. Asanother example, the processing module determines the retrieval optionto include retrieving the plurality of sets of encoded data slices whenthe retrieval reliability requirement indicates high reliability isrequired.

The method continues at step 222 where the processing module retrievesthe file, based on the retrieval option, in at least one of a legacyformat from the legacy storage system and a plurality of sets of encodeddata slices from the dispersed error coding storage system. Theprocessing module decodes the plurality of sets of encoded data slicesto produce reconstructed data when retrieving the plurality of sets ofencoded data slices and outputs the reconstructed data in accordancewith the retrieval option. Alternatively, or in addition to, theprocessing module decodes the plurality of sets of encoded data slicesto produce reconstructed data, compares the reconstructed data with thelegacy format of the file, and overwrites the legacy format of the filewith the reconstructed data when the legacy format of the file does notsubstantially match the reconstructed data. Alternatively, or inaddition to, the processing module decodes the plurality of sets ofencoded data slices to produce reconstructed data, compares thereconstructed data with the legacy format of the file, deletes thelegacy format of the file when the legacy format of the filesubstantially matches the reconstructed data, and updates a datalocation table to indicate that the file is not stored in the legacystorage system.

FIG. 13 is a flowchart illustrating an example of migrating data. Themethod begins with step 230 where a processing module (e.g., of adispersed storage (DS) unit) retrieves a first data copy from a firstmemory of a plurality of memories containing a plurality of copies ofthe data. For example, data is replicated using a redundant array ofindependent disks (RAID) approach such that a plurality of redundantcopies of the data are stored in the plurality of memories. The methodcontinues at step 232 where the processing module dispersed storageerror encodes the first data copy to produce encoded data slices. Themethod continues at step 234 where the processing module sends theencoded data slices to a dispersed storage network (DSN) memory forstorage therein.

The method continues at step 236 where the processing module determineswhether the encoded data slices are successfully stored in the DSNmemory. The determination may be based on one or more of a query,receiving a storage confirmation message, receiving an error message,and receiving a command. For example, the processing module determinesthat the encoded data slices are successfully stored when the processingmodule receives a slice storage confirmation message from a writethreshold number of DS units of the DSN memory. The method repeats backto step 230 when the processing module determines that the encoded dataslices are not successfully stored in the DSN memory. The methodcontinues to step 238 when the processing module determines that theencoded data slices are stored successfully in the DSN memory.

The method continues at step 238 where the processing module deletes allbut one copy of the plurality of copies of the data from the pluralityof memories. For example, the processing module deletes the redundantcopies of the data from the RAID system leaving just one copy in theRAID system in addition to the copy stored as encoded data slices in theDSN memory. In addition, the processing module may send a message to aRAID controller indicating that storing further redundant copies of thedata is not required (e.g., since the data is now backed up in the DSNmemory).

FIG. 14 is a flowchart illustrating another example of migrating data.The method begins with step 240 where a processing module (e.g., of amemory control module) determines error characteristics of a memory. Theerror characteristics include one or more of real-time stored errors,historical storage errors, retrieval errors, latency performance outsideof a specification, and a performance factor out of specification. Thedetermination of the error characteristics may be based on one or moreof receiving an error characteristic history, a message, a lookup, and acommand. The method continues at step 242 where the processing moduledetermines whether the error characteristics compare favorably to athreshold. The method repeats back to step 240 when the processingmodule determines that the error characteristics compare favorably to athreshold. The method continues to step 244 when the processing moduledetermines that the error characteristics does not compare favorably tothe threshold (e.g., too many errors).

The method continues at step 244 where the processing module assigns aportion of a responsible dispersed storage network (DSN) address rangeof the memory to at least one other memory. For example, the processingmodule assigns 10% of the DSN address range of the memory to a secondmemory by choosing the second memory (e.g., with available capacity) andmodifying a DSN address to physical location table to indicate that the10% of the DSN address range is now assigned to the second memory andnot to the memory.

The method continues at step 246 where the processing module migratesencoded data slices corresponding to the portion of the responsible DSNaddress range to the at least one other memory. For example, theprocessing module retrieves encoded data slices from the memorycorresponding to at least a portion of the 10% of the DSN address rangeand sends the encoded data slices to the second memory for storagetherein.

The method continues at step 248 where the processing module determineswhether all of the responsible DSN address range of the memory has beenreassigned and slices migrated. The determination may be based onexamination of the virtual DSN address to physical location table,wherein the table indicates whether slice names are no longer allocatedto the memory. The method repeats back to step 244 when the processingmodule determines that all of the responsible DSN address range of thememory has not been reassigned. For example, the method repeats backwhen the processing module determines that only 30% of the responsibleDSN address range of the memory has been reassigned. The methodcontinues to step 250 when the processing module determines that all theresponsible DSN address range of the memory has been reassigned. Forexample, the processing module determines that all of the responsibleDSN address range of the memory has been reassigned when 100% of theresponsible DSN address range of the memory has been reassigned.

The method continues at step 250 where the processing moduledecommissions the memory. In addition, the decommission memory may besubsequently replaced by new memory. In addition, the processing modulemay retrieve migrated encoded data slices from the at least one othermemory and store the encoded data slices in the new memory.

FIG. 15 is a flowchart illustrating an example of repurposing a memory.The method begins with step 252 where a processing module (e.g., of amemory control module) determines a memory for repurposing. Thedetermination may be based on one or more of detection of a new memorybeing added to a dispersed storage (DS) unit, detection of a legacymemory now installed in a DS unit and detection of a memory that isidentified for disposal. Repurpose them scenarios include one or more ofmoving a memory from a legacy system to a dispersed storage network(DSN) system, moving a memory from a DS unit to another DS unit,disposing of a memory from a legacy system, and disposing of a memoryfrom a DSN system.

The method continues at step 254 where the processing module determinesrepurposing requirements. The determination may be based on one or moreof a message, a scenario type (e.g., disposal or moving), a securityrequirement, a data type, a lookup, a predetermination, a message, and acommand. Repurposing requirements may include one or more of a requirednumber of write cycles indicator, whether to analyze the memory afterone or more write cycles, what type of repurposing data to write to thememory, and a maximum amount of time to utilize when repurposing thememory.

The method continues at step 256 where the processing module determineserror coding dispersal storage function parameters. The determinationmay be based on one or more of the memory selected for repurposing, amemory type indicator, a data type indicator, a security requirement,the repurposing requirements, a repurposing scenario indicator, alookup, a predetermination, a message, and a command. For example, theprocessing module determines a pillar width to be 32 when the securityrequirement indicates high security. As another example, the processingmodule determines the pillar width to be 6 when the security requirementindicates low security.

The method continues with step 258 where the processing moduledetermines repurposing data. The repurposing data includes data to bestored as encoded data slices in the memory. The determination may bebased on one or more of the error coding dispersal storage functionparameters, the repurposing requirements, a message, a repurposingscenario indicator, a memory type indicator, a data type indicator, asecurity requirement, a lookup, a predetermination, a message, and acommand. For example, the processing module determines repurposing datato include a random pattern of zeros and ones when the scenario typeindicates that the memory is being moved from a legacy system to a DSNsystem. As another example, the processing module determines repurposingdata to include all zeroes when the scenario type indicates that thememory is being disposed.

The method continues at step 260 where the processing module dispersederror encodes the repurposing data utilizing the error coding dispersalstorage function parameters to produce encoded repurposing data slices.The method continues at step 262 where the processing module sends theencoded repurposing data slices to the memory for storage therein. Themethod continues with at step 264 where the processing module determineswhether repurposing is complete. The determination may be based on oneor more of the error coding dispersal storage function parameters, therepurposing data, a required number of write cycles indicator, a numberof actual write cycles indicator, a query, a test, a read result, therepurposing requirements, a message, a repurposing scenario indicator, amemory type indicator, a data type indicator, a security requirement, alookup, a predetermination, a message, and a command. For example, theprocessing module determines that the repurposing is not complete whenthe number of actual write cycles indicator is less than the requirednumber of write cycles indicator. The method repeats back to step 262when the processing module determines that the repurposing is notcomplete. In such repeating, the processing module may write the samedata to the memory or write more data as specified by the repurposingdata. The processing module may write the repurposing data once, 10times, 100 times, or even more than 1,000 times to the memory. Note thatan improvement to data security may be provided when writing data overold data of the memory multiple times.

The method continues to step 266 when the processing module determinesthat the repurposing is complete. The method continues at step 266 wherethe processing module indicates that the repurposing is complete (e.g.,by sending a message). In addition, the processing module may allocatethe memory for utilization within the DSN system when not disposing ofthe memory. The processing module deactivates the memory prior todisposing of the memory.

As may be used herein, the terms “substantially” and “approximately”provides an industry-accepted tolerance for its corresponding termand/or relativity between items. Such an industry-accepted toleranceranges from less than one percent to fifty percent and corresponds to,but is not limited to, component values, integrated circuit processvariations, temperature variations, rise and fall times, and/or thermalnoise. Such relativity between items ranges from a difference of a fewpercent to magnitude differences. As may also be used herein, theterm(s) “operably coupled to”, “coupled to”, and/or “coupling” includesdirect coupling between items and/or indirect coupling between items viaan intervening item (e.g., an item includes, but is not limited to, acomponent, an element, a circuit, and/or a module) where, for indirectcoupling, the intervening item does not modify the information of asignal but may adjust its current level, voltage level, and/or powerlevel. As may further be used herein, inferred coupling (i.e., where oneelement is coupled to another element by inference) includes direct andindirect coupling between two items in the same manner as “coupled to”.As may even further be used herein, the term “operable to” or “operablycoupled to” indicates that an item includes one or more of powerconnections, input(s), output(s), etc., to perform, when activated, oneor more its corresponding functions and may further include inferredcoupling to one or more other items. As may still further be usedherein, the term “associated with”, includes direct and/or indirectcoupling of separate items and/or one item being embedded within anotheritem. As may be used herein, the term “compares favorably”, indicatesthat a comparison between two or more items, signals, etc., provides adesired relationship. For example, when the desired relationship is thatsignal 1 has a greater magnitude than signal 2, a favorable comparisonmay be achieved when the magnitude of signal 1 is greater than that ofsignal 2 or when the magnitude of signal 2 is less than that of signal1.

As may also be used herein, the terms “processing module”, “module”,“processing circuit”, and/or “processing unit” may be a singleprocessing device or a plurality of processing devices. Such aprocessing device may be a microprocessor, micro-controller, digitalsignal processor, microcomputer, central processing unit, fieldprogrammable gate array, programmable logic device, state machine, logiccircuitry, analog circuitry, digital circuitry, and/or any device thatmanipulates signals (analog and/or digital) based on hard coding of thecircuitry and/or operational instructions. The processing module,module, processing circuit, and/or processing unit may have anassociated memory and/or an integrated memory element, which may be asingle memory device, a plurality of memory devices, and/or embeddedcircuitry of the processing module, module, processing circuit, and/orprocessing unit. Such a memory device may be a read-only memory, randomaccess memory, volatile memory, non-volatile memory, static memory,dynamic memory, flash memory, cache memory, and/or any device thatstores digital information. Note that if the processing module, module,processing circuit, and/or processing unit includes more than oneprocessing device, the processing devices may be centrally located(e.g., directly coupled together via a wired and/or wireless busstructure) or may be distributedly located (e.g., cloud computing viaindirect coupling via a local area network and/or a wide area network).Further note that if the processing module, module, processing circuit,and/or processing unit implements one or more of its functions via astate machine, analog circuitry, digital circuitry, and/or logiccircuitry, the memory and/or memory element storing the correspondingoperational instructions may be embedded within, or external to, thecircuitry comprising the state machine, analog circuitry, digitalcircuitry, and/or logic circuitry. Still further note that, the memoryelement may store, and the processing module, module, processingcircuit, and/or processing unit executes, hard coded and/or operationalinstructions corresponding to at least some of the steps and/orfunctions illustrated in one or more of the Figures. Such a memorydevice or memory element can be included in an article of manufacture.

The present invention has been described above with the aid of methodsteps illustrating the performance of specified functions andrelationships thereof. The boundaries and sequence of these functionalbuilding blocks and method steps have been arbitrarily defined hereinfor convenience of description. Alternate boundaries and sequences canbe defined so long as the specified functions and relationships areappropriately performed. Any such alternate boundaries or sequences arethus within the scope and spirit of the claimed invention. Further, theboundaries of these functional building blocks have been arbitrarilydefined for convenience of description. Alternate boundaries could bedefined as long as the certain significant functions are appropriatelyperformed. Similarly, flow diagram blocks may also have been arbitrarilydefined herein to illustrate certain significant functionality. To theextent used, the flow diagram block boundaries and sequence could havebeen defined otherwise and still perform the certain significantfunctionality. Such alternate definitions of both functional buildingblocks and flow diagram blocks and sequences are thus within the scopeand spirit of the claimed invention. One of average skill in the artwill also recognize that the functional building blocks, and otherillustrative blocks, modules and components herein, can be implementedas illustrated or by discrete components, application specificintegrated circuits, processors executing appropriate software and thelike or any combination thereof.

The present invention may have also been described, at least in part, interms of one or more embodiments. An embodiment of the present inventionis used herein to illustrate the present invention, an aspect thereof, afeature thereof, a concept thereof, and/or an example thereof. Aphysical embodiment of an apparatus, an article of manufacture, amachine, and/or of a process that embodies the present invention mayinclude one or more of the aspects, features, concepts, examples, etc.described with reference to one or more of the embodiments discussedherein. Further, from figure to figure, the embodiments may incorporatethe same or similarly named functions, steps, modules, etc. that may usethe same or different reference numbers and, as such, the functions,steps, modules, etc. may be the same or similar functions, steps,modules, etc. or different ones.

While the transistors in the above described figure(s) is/are shown asfield effect transistors (FETs), as one of ordinary skill in the artwill appreciate, the transistors may be implemented using any type oftransistor structure including, but not limited to, bipolar, metal oxidesemiconductor field effect transistors (MOSFET), N-well transistors,P-well transistors, enhancement mode, depletion mode, and zero voltagethreshold (VT) transistors.

Unless specifically stated to the contra, signals to, from, and/orbetween elements in a figure of any of the figures presented herein maybe analog or digital, continuous time or discrete time, and single-endedor differential. For instance, if a signal path is shown as asingle-ended path, it also represents a differential signal path.Similarly, if a signal path is shown as a differential path, it alsorepresents a single-ended signal path. While one or more particulararchitectures are described herein, other architectures can likewise beimplemented that use one or more data buses not expressly shown, directconnectivity between elements, and/or indirect coupling between otherelements as recognized by one of average skill in the art.

The term “module” is used in the description of the various embodimentsof the present invention. A module includes a functional block that isimplemented via hardware to perform one or module functions such as theprocessing of one or more input signals to produce one or more outputsignals. The hardware that implements the module may itself operate inconjunction software, and/or firmware. As used herein, a module maycontain one or more sub-modules that themselves are modules.

While particular combinations of various functions and features of thepresent invention have been expressly described herein, othercombinations of these features and functions are likewise possible. Thepresent invention is not limited by the particular examples disclosedherein and expressly incorporates these other combinations.

1. A method comprises: receiving a retrieval request for a file;determining whether the file is being migrated from a legacy storagesystem to a dispersed error coding storage system; when, the file isbeing migrated from the legacy storage system to the dispersed errorcoding storage system, determining a retrieval option for the file; andbased on the retrieval option, retrieving the file in at least one of: alegacy format from the legacy storage system; and a plurality of sets ofencoded data slices from the dispersed error coding storage system. 2.The method of claim 1, wherein the determining whether the file is beingmigrated further comprises: determining whether that the file is storedin a legacy format in the legacy storage system and is stored as aplurality of sets of encoded data slices in the dispersed error codingstorage system.
 3. The method of claim 1, wherein the retrieval optioncomprises at least one of: retrieve the legacy format from the legacystorage system only; retrieve the plurality of sets of encoded dataslices from the dispersed error coding storage system only; and retrieveboth the legacy format from the legacy storage system and the pluralityof sets of encoded data slices from the dispersed error coding storagesystem and outputting, based on first available, the legacy format orthe plurality of sets of encoded data slices.
 4. The method of claim 3further comprises: when retrieving the plurality of sets of encoded dataslices, decoding the plurality of sets of encoded data slices to producereconstructed data; and outputting the reconstructed data in accordancewith the retrieval option.
 5. The method of claim 1 further comprises:decoding the plurality of sets of encoded data slices to producereconstructed data; comparing the reconstructed data with the legacyformat of the file; and overwriting the legacy format of the file withthe reconstructed data when the legacy format of the file does notsubstantially match the reconstructed data.
 6. The method of claim 1further comprises: decoding the plurality of sets of encoded data slicesto produce reconstructed data; comparing the reconstructed data with thelegacy format of the file; deleting the legacy format of the file whenthe legacy format of the file substantially matches the reconstructeddata; and updating a data location table to indicate that the file isnot stored in the legacy storage system.
 7. The method of claim 1,wherein the determining the retrieval option for the file is based on atleast one of: a retrieval latency requirement; a retrieval reliabilityrequirement; a memory availability requirement; and a maximum bandwidthutilization requirement.
 8. A computer comprises: an interface; amemory; and a processing module operable to: receive, via the interface,a retrieval request for a file; determine whether the file is beingmigrated from a legacy storage system to a dispersed error codingstorage system; when, the file is being migrated from the legacy storagesystem to the dispersed error coding storage system, determine aretrieval option for the file; and based on the retrieval option,retrieve, via the interface, the file in at least one of: a legacyformat from the legacy storage system; and a plurality of sets ofencoded data slices from the dispersed error coding storage system. 9.The computer of claim 8, wherein the processing module functions todetermine whether the file is being migrated further by: determiningwhether that the file is stored in a legacy format in the legacy storagesystem and is stored as a plurality of sets of encoded data slices inthe dispersed error coding storage system.
 10. The computer of claim 8,wherein the retrieval option comprises at least one of: retrieve thelegacy format from the legacy storage system only; retrieve theplurality of sets of encoded data slices from the dispersed error codingstorage system only; and retrieve both the legacy format from the legacystorage system and the plurality of sets of encoded data slices from thedispersed error coding storage system and outputting, based on firstavailable, the legacy format or the plurality of sets of encoded dataslices.
 11. The computer of claim 10, wherein the processing modulefurther functions to: when retrieving the plurality of sets of encodeddata slices, decode the plurality of sets of encoded data slices toproduce reconstructed data; and output, via the interface, thereconstructed data in accordance with the retrieval option.
 12. Thecomputer of claim 8, wherein the processing module further functions to:decode the plurality of sets of encoded data slices to producereconstructed data; compare the reconstructed data with the legacyformat of the file; and overwrite, via the interface, the legacy formatof the file with the reconstructed data when the legacy format of thefile does not substantially match the reconstructed data.
 13. Thecomputer of claim 8, wherein the processing module further functions to:decode the plurality of sets of encoded data slices to producereconstructed data; compare the reconstructed data with the legacyformat of the file; delete the legacy format of the file when the legacyformat of the file substantially matches the reconstructed data; andupdate a data location table to indicate that the file is not stored inthe legacy storage system.
 14. The computer of claim 8, wherein theprocessing module functions to determine the retrieval option for thefile based on at least one of: a retrieval latency requirement; aretrieval reliability requirement; a memory availability requirement;and a maximum bandwidth utilization requirement.